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Abstract 

This paper presents APPRAISALWeb, a 2-year Research & Development project jointly funded by the Universidad Politecnica 
de Valencia and the regional government of Valencia (Spain), and developed by the CAMILLE Research Group at that 
university. In general terms, the project aims to suggest a comprehensive methodology for the pedagogical analysis of the Web as 
an environment for language learning. More specifically, a major objective of the project is to develop an online interactive 
platform that serves as a tool to analyze, classify and evaluate Web-based resources for language learning from a pedagogical 
point of view. As CALL (Computer Assisted Language Learning) research literature suggests, it is necessary to conduct a 
thorough pedagogical analysis of the Web and its CALL resources if we are to take full advantage of its many possibilities. Thus, 
the project meets two fundamental needs that have been extensively expressed in the area of CALL: the need for theoretical 
models and methodologies and the necessity of specific tools for analysing and searching for available Web-based CALL 
resources. First, the context of the project will be presented. Then the methodology behind APPRAISALWeb will be discussed. 
Finally, conclusions will be drawn from work in progress during the first stage of the project. 
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1. Introduction 

In the current context of technology-assisted learning, it is increasingly important to base the implementation of 
educational technology, regardless of its nature and format, on sound pedagogical principles and criteria, if we want 
to take full advantage of its educational potential. This general principle is particularly relevant within the relatively 
innovative field of Web-based Computer Assisted Language Learning (CALL). It becomes therefore necessary to 
undertake a profound analysis of the Web as a language learning environment, not only due to the novelty of 
associated technologies (social networks, virtual worlds, Computer Mediated Communication, etc.), but also and 
especially because of the existence of highly heterogeneous contents and the usual lack of quality control for such 
contents, among other drawbacks (Felix 2003). Consequently, this paper presents a research project related to the 
pedagogical analysis of Web-based CALL named APPRAISALWeb ( Analisis Pedagogicos de Recursos de 
Aprendizaje Interactive) y Sistemas de Aprendizaje de Lenguas a traves de la Web, i.e. Pedagogical Analysis of 
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Web-based Language Learning Resources and Systems). The project is led by the CAMILLE research group 
(Computer Assisted Multimedia Interactive Language Learning Environments), at the Universidad Politecnica de 
Valencia, in Spain. 

The aim of the project APPRAISALWeb is the development, implementation and social dissemination, on the 
one hand, of an efficient methodology to observe, analyse and evaluate Web-based Language Learning resources 
(for English, French and Catalan as a starting point) by applying pedagogical criteria, and, on the other hand, of a 
knowledge base called APPRAISALWEB (built on the work of a previous database called WIRESLAB 1 ), which 
will allow users to catalogue, describe and pedagogically analyse CALL resources currently available on the Web. 
In this way, a key objective would be to study both the educational possibilities of this environment in language 
learning and teaching, and the real picture of Web-based CALL through the actual pedagogical analysis and 
evaluation of currently available online resources. 

The suggested methodology, based on a theoretical framework of pedagogical evaluation, aims at the 
development of a solid tool for the retrieval, classification and interpretation of pedagogical information on Web- 
based educational resources, consisting of a platform located on a website, which collects data to assess pedagogical 
and didactic issues of such resources. Since the foundations of this tool (the database called WIRESLAB, also 
developed within the CAMILLE research group by Rafael Seiz in 2006) are already set, one of the purposes of the 
present project is its thorough development and improvement so as to build up an online global knowledge base 
(APPRAISALWEB) dealing with pedagogical aspects of Web-based CALL resources, which is, in turn, openly 
available on a website for teachers, learners, researchers and developers. Both the pedagogical evaluation 
methodology and the tool will therefore assist the academic community and the society in the tasks of efficient 
search, classification, retrieval and use of language learning resources through the Web. 

In this presentation of the project APPRAISALWeb, first, the context where it was developed will be discussed, 
and, then, its relevance as well as theoretical and practical basis, including objectives, purpose and expected results 
will be presented. Later on, some issues about the project methodology will be dealt with, since a brief discussion on 
such methodology could be of use in similar attempts to analyse the Web in effective language learning. In this 
section, we would also like to invite the involvement of the academic, teaching and research communities in the 
project’s objectives and outcomes 11 . Finally, some proposals are included for further research and follow-up of the 
work initiated in this project. 

2. The APPRAISALWeb project 

APPRAISALWeb is a R&D project carried out by the research group CAMILLE 111 from the Universidad 
Politecnica de Valencia, which has been devoted for more than two decades to research on CALL, including a line 
of research on Web-based CALL. During the winter semester of the academic year 2009-10, the project obtained 
financial support both from the Universidad Politecnica de Valencia and the regional government of Valencia 
(Generalidad Valenciana), within the framework of their support to emerging research groups. The duration of the 
project is 2 years, starting in December 2009, so the tasks stated in its work plan are well under development. The 
research team consists of seven members: four researchers that belong to the teaching staff at the University’s 
Applied Linguistics Department, a research technician (with a double professional background as computer expert 
and academic researcher), a research assistant (computer technician trainee) and a research team coordinator with 
fully academic background. As a consequence, it is a multidisciplinary team that works in an independent yet 
coordinated way on the multifaceted aspects related with the twofold nature of the project, involving both 
computing and academic (mainly pedagogical) issues. This coordinated teamwork is carried out following the 
principles of the methodology called template approach as suggested for research and development work within the 
CALL domain by Ana Gimeno Sanz (2002), whereby team members with varying backgrounds and expertise work 
on different but related issues in an autonomous and coordinated way, thus providing each other with valuable 
feedback. 

The CAMILLE research group has been working since 1993 on the development, implementation and evaluation 
of CALL resources and tools, both from a pedagogical and technical standpoint. It should be noted that CAMILLE 
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is interested in dealing with any aspect and phase of the process of CALL teaching and learning, from an 
interdisciplinary and global point of view: pedagogical and computer design, didactic implementation, materials 
development, evaluation and pedagogical analysis. Previous projects are focused on some of these aspects. 
APPRAISALWeb falls within this context, this time dealing with pedagogical analysis and evaluation of Web-based 
CALL, and will benefit from the work of previous projects, giving valuable feedback on former and future projects 
as well. 

Within the field of language learning, the Web contains a huge amount of educational resources covering a wide 
range of contexts, formats, levels and pedagogical purposes. Current methods for information search and retrieval 
(the most common of which is Web search engines, particularly Google) do not allow users to conduct advanced 
searches of CALL resources based on pedagogical criteria and characteristics. They are of no use for the purpose of 
pedagogically analyzing and evaluating those resources, once they have been found, either. The APPRAISALWeb 
project, on the contrary, aims at meeting those demands, and to do so it develops: (1) a theoretical model for 
pedagogical evaluation; (2) a methodology for pedagogical analysis; and (3) an online computer tool 
(APPRAISALWeb) which is dynamic, open and interactive in order to implement that analysis of Web-based CALL 
resources, from a pedagogical perspective. 

As can be seen in previous CALL literature, former theoretical models (Zhao 1996, Godwin- Jones 1999, Felix 
2001, 2002, 2003, Hampel and Baber 2003, Chapelle 2001) that could be used to lay the foundations of the 
pedagogical evaluation suggested here, do not incorporate, in our view, the type of global, comprehensive and 
detailed view of the language learning process that we are willing to take in our approach. In the state-of-the-art of 
CALL nowadays there is not any resource or tool for retrieving and analysing pedagogical data of Web-based 
CALL as exhaustive, open and dynamic as the one suggested here. 

The existence and availability of a Web-based resource (APPRAISALWeb) that is able to pedagogically 
catalogue, describe, analyse and evaluate language learning courses and resources on the Web, as well as conduct 
advanced searches of educational resources based on pedagogical criteria will be very useful to improve the 
knowledge of the potential of this environment and technology for language learning, to enhance the efficient 
pedagogical use of the resources, and to foster the reusability of language learning resources and Learning Objects 
available on the Web. The open nature of such resource and knowledge base (including the possibility of continuous 
updates and contributions from the academic and scientific community) will also enable the implementation of the 
concept of distributed knowledge. This will render benefits to the vast community of people interested in language 
learning: teachers, learners, researchers, developers, company representatives, publishing companies and 
educational institutions. The tools that will result from the project will also become interesting in the context of the 
common European framework of Higher Education, since they will enable European citizens (and also outside the 
continent) to use the Web efficiently for language (self-)leaming and teaching. The initial hypotheses for this work 
are: (1) the Web is an environment where language learning and teaching can take place efficiently; (2) the Web 
implies some kind of innovation in language learning; and (3) to meet the previous demands, it is necessary to 
evaluate the resources pedagogically, which will be possible only if solid pedagogical criteria based on previous 
research are applied. 

The general objective of the project, namely the pedagogical analysis of the Web as a feasible environment for 
efficient language learning, is then broken down in the following aspects, which are approached in a global manner: 
(1) to suggest a methodological model for the pedagogical analysis of Web-based CALL; (2) to develop a tool for 
analysing Web-based CALL resources from a pedagogical standpoint, consisting of a resource or knowledge base 
(APPRAISALWeb), based on a database (WIRESLAB), which is based on pedagogical criteria, easy to use, allows 
users to enter data in a collaborative way and is available to a large number of people through the Web; (3) to get to 
know and be able to use efficiently the Web environment and its language learning resources as an integrated part of 
the language learning process; and (4) to develop an open web-enabled resource, i.e. an interactive platform for the 
retrieval and processing of pedagogical data on Web-based CALL resources, potentially useful for several languages 
and a wide range of users, which will also benefit society as a whole and could be used in future R&D projects. 


670 


Rafael Seiz-Ortiz et al. / Procedia Social and Behavioral Sciences 15 (2011) 667-671 


3. Methodology 

The project incorporates both a theoretical and a practical side. On the theoretical side, a comprehensive state-of- 
the-art review of the CALL and CALL-related literature is carried out, in search for concrete pedagogical 
parameters that can inform the theoretical model and the computer-based evaluation tools. Part of such work has 
already been done within the CAMILLE research group (Seiz Ortiz 2006). At a more practical level, the project 
will: (1) develop computer tools (a database and a knowledge base, APPRAISALWeb, available on a website) in 
order to carry out the pedagogical analysis and evaluation of Web-based CALL resources; (2) build up a significant 
corpus of language learning resources on the Web (for English, Catalan and French); and (3) evaluate such a corpus 
by means of the previously developed model and the tools. 

Due to the fully interdisciplinary nature of the project, teams will cooperate that belong to fields as diverse as 
computer science and applied linguistics, including researchers and teachers. Therefore, these groups will work 
partly independently and partly in a highly coordinated way. Computer experts will implement and develop the 
computer tools whose conceptual and pedagogical content will be carried out by teachers and researchers, and, 
therefore, direct and continuous contact between the two groups will be kept at all times. 

At the final stages of the project, when the practical implementation, use and evaluation of the tool will take 
place, participation will be required of external people, such as learners, teachers, other computer experts and 
researchers. Contributions made by these people will be incorporated, through forms and questionnaires especially 
designed to that end, into the project’s website, from where access passwords will be assigned to the new users that 
will be responsible for evaluating both the platform and the language learning resources. For each one of the 4 
groups mentioned above, a participation of between 20 and 40 people will be considered. Results from such 
evaluation of the APPRAISALWeb resource will be incorporated into the final version of the project website, when 
it becomes openly and freely available to the general public. 

The work plan of the project has been established in 2 years and 3 general stages, distributed in the following 
way. During the first year, stage 1 of the project is undertaken, that is, the proposal of a theoretical model and the 
development of the tool WIRESLAB (the database), and stage 2 makes a start, consisting of the development of the 
online pedagogical evaluation platform called APPRAISALWeb. In the course of the second year of the project, 
stage 2 is completed, and stage 3 is carried out, with the practical implementation and evaluation of the tool 
APPRAISALWeb itself, as well as the dissemination of the projects outcomes. 

4. Results of the project and future work 

The project aims at the pursuit of a series of concrete results that will be of great academic and social interest. 
The most visible outcome is the APPRAISALWeb website lv , an open and interactive platform that will consist of a 
huge database, a knowledge base and evaluation centre for Web-based CALL, which will incorporate repositories of 
Web-based learning resources for different languages and will include advanced search facilities based on language 
learning pedagogical criteria. At a later stage in our research work, it will also incorporate a comprehensive and 
complete knowledge base devoted to the pedagogical study of the web for language learning. But, former stages 
such as the specialized research literature review and the proposal of an improved version of a theoretical 
framework for the exhaustive pedagogical analysis of Web-based CALL, can also be of interest for the profession. 

Regarding the project’s dissemination, on the one hand, results and conclusions will be presented in several 
publications in scientific journals, professional magazines, online forums, blogs, discussion lists and similar 
professional communications facilities available on the Web. Outcomes will also be presented at specialized 
conferences. There will also be a publication at the end of the project in the form of a book summarizing the most 
remarkable academic issues raised by the project work, including reflections and insights that could be of help to 
similar forthcoming projects. Training activities will also be organised, such as workshops or seminars related with 
the use and implementation of the APPRAISALWeb resource and related tools, addressed at professionals interested 
in CALL. 
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We believe that the potential social interest of the project could result in agreements with companies and public 
or private institutions, in order to disseminate the results, methodologies and other relevant outcomes of the 
APPRAISALWeb project. Needless to say, researchers are welcome to participate through the tool website, 
submitting comments, suggestions and evaluations at a later stage of the project, when the website is fully 
operational. 

5. Conclusions 

Since this paper presents work which is under development, too definitive conclusions cannot be drawn yet. 
Nevertheless, we wish to point out the twofold need of suggesting methodologies and models solidly based in 
previous research and pedagogical criteria, when analyzing what the innovation of the Web can provide in the field 
of language learning, on the one hand, and, on the other hand, of developing efficient and easy-to-use tools to search 
for, classify, retrieve and reuse readily language learning resources available on the Web, thereby taking full 
advantage of the educational possibilities of such environment. Moreover, our proposal is to do so by carrying out 
effective multidisciplinary teamwork, the only suitable method when dealing with educational technology in 
general. 

The project presented here has a remarkable social significance and interest, since it may assist all those 
interested in language learning (students, teachers, developers and researchers) in the efficient and pedagogical use 
of interactive language learning resources available nowadays on the Web, which otherwise would be difficult to be 
found, classified and used by the society. 
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